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Introduction: Towards a Problem-Solving Instruction 

This talk is focussed on the identification and evaluation of facets of problem solving 
instruction (PSI) in the natural setting of German Mathematics classrooms. It is 
supposed to give an impression of the way our research team at the Berlin Max-Planck 
Institute for Human Development is trying to link the TIMS-Study and the TIMS- Video- 
Study to create a network of explanatory power that goes beyond the international 
achievement horserace. 



What is PS? 

PSI can be regarded as the critical joint between instructional research and modem 
learning theories of cognitive psychology. PSI goes beyond simply confronting students 
with well-defined addition problems, multiplication problems, fraction problems or word 
problems as it is done in any Mathematics classroom. 

The specific use of the term "problem" in the context of PSI needs to be clarified: Its early 
roots can be identified in the philosophical works of Dewey, who speaks of problems as 
"forked-road-situations" that call on human curiosity as a natural resource. Problems 
can be roughly described as "nonroutine questions of some complexity". They include a 
non-desired starting situation and a desired goal with the actual means to the specific 
end being initially unknown to the person faced with it. A problem in itself carries a gap, 
a disturbance or an incongmity which stands between the question and it's solution. 
These gaps that make a problem "nonroutine" or "ill-stmctured" are supposed to have a 
motivating and challenging function on the individuals confronted with them. 

Schoenfeld (1989) describes "problem" in the context of Mathematics instmction as "a 
task (a) in which the student is interested and engaged and for which he wishes to 
obtain a resolution, and (b) for which the student does not have a readily accessible 
Mathematical means by which to achieve that resolution" (p. 88). 

Different Research Frameworks and Conceptualizations of PSI 

This is not the place to fully acknowledge the body of research that has been 
accumulated on PSI. The following quotations are meant to illustrate how different 
researchers stress different facets of PSI in Mathematics. Problem solving instruction 
can be differently understood from different theoretical approaches. 
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Hiebert, Carpenter, Fennema et al. (1996) are relating to the works of Dewey and 
postulate to incorporate problem solving in the classroom. "Students should be allowed 
to make the subject problematic. We argue that this single principle captures what is 
essential for instructional practice." Palincsar (1998) stresses the role of the teacher as a 
mediator, a promoter of discussion among pupils in the context of a constructivistically 
oriented instruction. "The crucial role that the teacher plays in promoting the co- 
construction of knowledge in classrooms was also demonstrated in the research [...] (on) 
the dynamic role of the teacher in guiding classroom discussions in the context of 
Mathematical problem solving." The theoretical works on PSI found their way into the 
Mathematics curriculum in the U.S.: In the 'curriculum and evaluation standards for 
school Mathematics' released by the National Council of Teachers of Mathematics 
(NCTM, 1989) PSI is given a prominent role: "Problem solving is the process by which 
students experience the power and usefulness of mathematics in the world around them. 
It is also a method of inquiry and application, interwoven throughout the Standards to 
provide a consistent context for learning and applying mathematics. Problem situations 
can establish a "need to know" and foster the motivation for the development of 
concepts." The ciurent draft version 'principles and standards for school Mathematics' 
('Standards 2000'; NCTM, 1998) puts an even larger weight on the implementation of 
PSI in Mathematics classrooms. 

The NCTM Guidelines influenced educational practice not only in the U.S. In the 'Open- 
Ended Approach' Japanese educators blended central elements of the NCTM approach 
with Japanese ideas of PSI (Becker & Shimada, 1997). Results from the TIMSS-Video- 
Study show that "Japanese Mathematics teachers systematically use alternative 
representations and student-developed solution methods to reveal the Mathematical 
problem" (Baumert, Lehmann et al., 1997; authors translation). Nohda (1995) underlines 
the importance of confronting students with open-ended Mathematical problems. These 
are problems that allow the use of alternative solution methods that can lead to correct 
solutions on different levels of complexity: "In Japan, new approaches to problem solving 
are being emphasized in school Mathematics. [...] (The teaching approach using "open- 
ended problems") has been shown to be the most effective in fostering both students' 
Mathematical thinking and students' motivation toward Mathematics." 
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All these researchers unanimously promote problem solving as a necessary, if not crucial 
component of a cognitively demanding instruction. But the exact nature of the relation 
between (1) problem solving as an overt, organized classroom activity, (2) problem 
solving as a t 3 T)e of cognitive processing and (3) problem solving as a more or less 
generabzed competency often is not made clear. 

Research Questions 

Empirical research from the different PSI frameworks described above has been focussed 
on experimental small group settings and interpretative microanalyses of the learning 
process. The question to what degree facets of PSI apply in the natural environments of 
everyday Mathematics classrooms and in what way these facets interact with the other 
featimes of instructional quality has been seldomly addressed. Groal of this study is to 
examine the nature of problem solving as a construct in instructional research using 
high inferential video ratings for nine different facets of PSI which where derived from 
research literatime. In oim exploration of problem solving as a construct we try to answer 
the following questions. 

Is it possible to define different facets of the construct PSI and to empirically identify 
them? How frequently do the different facets of Problem Solving Instruction apply in the 
natural setting of everyday German Mathematics classrooms? Can the different facets be 
integrated into a unidimensional instructional characteristic? Is there a unique specific 
problem solving variance which is distinct from other characteristics of Teacher 
Effectiveness? Can some elements of Teacher Effectiveness be identified as meaningful 
conditions for a Problem Solving Instruction? Does Problem Solving Instruction directly 
improve learning in Mathematics? In what way are the facets of problem solving 
instruction related to classroom discoimse and structimal characteristics of instruction? 
Do the external video ratings based on a small video sample correspond to the students' 
general view of instruction? 
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Study Design 

Nine facets of problem solving instruction are derived from different approaches in 
research literature. To assess these nine PS facets a video rating form is developed. This 
rating form is used in combination with a classroom observation inventory developed by 
Helmke, Schrader and Weinert (Weinert & Helmke, 1997), in which classic constructs of 
instructional quality such as teacher control, clarity of rules, organization and time-on- 
task are measured. The videotaped 8“* grade Mathematics lessons rated, are part of the 
German TIMS-Video Sample (Third International Mathematics and Science Study; 
Beaton et al., 1996; Baumert et al., 1997). Since the videotaped classes also participated 
in the TIMSS-Main study, this study provides us with curricular valid achievement 
scores and video recordings of up to three consecutive lessons. The resulting ratings of 
instruction are related to reference data from TIMSS, including student achievement 
and motivation (TIMSS and German longitudinal enhancement), low inference codes 
(TIMS-Video) and student perceptions of instruction (TIMSS-Student Background 
Questionnaire). 



Nine Facets of Problem Solving Instruction 

The following nine facets are meant to represent possible realization of problem solving 
in Mathematics instruction: 

1. Problem Solving Processes 

(Reference: traditional research on problem solving) 

2. Focusing 

(Reference: Japanese open-ended approach) 

3. Teacher as a Mediator 

(Reference: Japanese open-ended approach; constructivism) 

4. Openness of problems and solutions 

(Reference: Japanese open-ended approach; constructivism) 

5. Anchoring in student knowledge 
(Reference: constructivism) 

6. Student Discussion 
(Reference: constructivism) 

7. Student Cooperation 

(Reference: constructivism; Situated Learning) 

8. Multiple Contexts 
(Reference: Situated Learning) 

9. Authentic real-world embedding 
(Reference: Situated Learning) 




6 



- 6 - 



The facet "PS Processes" is derived from traditional psychological research on problem 
solving. In this facet we want to assess to what degree an instruction can be seen as an 
explicit goal-directed, multi-step PS process including evaluation and feed-back steps. 
"Focussing" describes the teacher’s concentrating on main goals or questions in the 
course of classroom interaction. The facet "Teacher as a Mediator" assesses whether the 
teacher is functioning as a mediator or promoter of the exchange of students ideas in 
discussions. The facet "Openness of Tasks and Solution Approaches" asks whether there 
are more than one possible correct solutions for the tasks and problems given. 
"Anchoring in Student Knowledge" assesses the degree to which a teacher tries to 
connect the given problems to Mathematical knowledge the students learned in previous 
lessons or courses. The facet "Student Discussion" asks whether the students are given 
the chance to negotiate Mathematical concepts and meanings. Another facet, "Student 
Cooperation" is focussed on whether students can work together in small groups on the 
given problems. "Multiple Contexts" refers to whether the teacher presents problems in 
more than one embedding, e.g. in different Mathematical contexts or using different 
media. The facet "Authentic Real-World Embedding" assesses the use of authentic 
problem situations or real-world problems. The following table (Table 1) shows some 
examples from oirr observation instrument. 

Table 1: Example for the Problem Solving Item "Teacher as a Mediator" 





^ ^ „ Does Does Does 

oes u y partially partially not absolutely 

^ apply aplly not apply 


3. Teacher as a mediator 

(Reference: Japanese open-ended approach; 

constructivism) 


n 


□ 


□ 


□ 


The teacher gives time to the students to develop 
ideas and find answers. 




The teacher puts the ideas of the students in 
relation to the context of the lesson. 


The teacher puts the contributions of the students 
in relation to each other. 


The teacher supports the students in formulating 
their ideas. 


If a student formulates incomplete and unclear 
ideas, the teacher investigates them without 
judging the students performance. 


The teacher doesn’t correct immediately every 
mistake. 
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Technical Details 

The German TIMSS Video-Sample consist of 1st Lessons from 82 classes, of which 34 
have 2nd lessons of which 31 have 3rd lessons, resulting in a total number of 147 
lessons. Each lesson was judged by two raters, so that 294 ratings were completed. The 
rating instrument includes 9 Items on Problem Solving and 36 classical Teacher 
Effectiveness Items. The video ratings were completed by trained raters. Training 
involved thorough discussion of the rating instrument and an extensive practicing phase 
in which several demo lessons where rated. Each video was judged independently by two 
raters. Each possible pair of raters saw one third of the lessons with each trisection of 
the sample balanced according to type of school. After watching a complete videotaped 
lesson the raters indicated presence or absence for each of the 45 items on a 4-point 
rating scale (1 'does not apply' to 4 'does fully apply'). Aggregated item scores for each 
video were computed on the basis of averaged ratings by the two independent judges. 
Averaged weighted kappa between raters was .77. 

Results 

Our first research question was how frequently the different facets of Problem Solving 
Instruction apply in the natural setting of everyday German Mathematics classrooms. 
Figure 1 shows that six of the nine facets can be regarded as rather rare events in 
German Mathematics instruction. Only the top three facets, PS Processes, Focussing 
and Teacher as a Mediator have approximately normal distributions. 
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Problem Solving 
Processes 




Openness of Tasks / 
Solution Methods 





Figure 1: Distributions of the nine Problem Solving Facets 



BEST COPY AVAILABLE 




9 



Proportion per Bar Proportion per Bar Proportion per Bar 



-9- 



Can the 9 facets be considered as realizations of a unidimensional construct? Table 2 
shows the resulting loading matrix of a varimax rotated principal component analysis. 

Table 2: Varimax-rotated Principal Component Analysis of the nine PS Facets 







Factors 






1 


2 


3 




''Constructivist 

Instruction" 


"Problem Processing" 


"Real-World 

Embedding" 




Argumentative 
reflective interaction 


Goal-oriented and 
focussed solution 
approach 


Problems 

in multiple natural 
contexts 


Student Discussion 


0,84 






Teacher as Mediator 


0,84 






Openness 


0,81 






Student Cooperation 


0,80 






Focusedness 




0,92 




Problem Solving Process 




0,64 


0,45 


Knowledge Anchoring 
Authentic Embedding 


0,51 


0,58 


0,92 


Multiple Contexts 






0,75 



Loadings under 0,40 are omitted 



PSI Facets 

The analysis brought up three factors with eigenvalues greater 1. Looking at the loading 
coefficients, three main components are separated: 

(I) argumentative and reflective interaction ("Constructivist Instruction") 

(II) goal-oriented and focussed solution approaches ("Problem Processing") 

(III) problems in multiple natural contexts ("Real-World Embedding") 

The second component is a httle less distinct than the other two considering the side 
loadings. We cannot say that the facets represent a single unidimensional construct. 



How do the problem solving facets relate to classic dimensions of teacher effectiveness? 
The following table 3 shows another factor analysis in which we analyzed the PS facets 
together with the 36 other observation items. 
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Table 3: Factor Analysis Varimax-rotated PCA Effectiveness Dimensions and PSI Facets 
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ORGANISATION 


0,87 


CONTROL 


0,86 


RULECLARITY 


0,84 


TIMEONTASK 


0,81 


SCONTINGENCY 


0,78 


ACH.-CLIMATE 


0,75 


SPEED 


0,56 


CONCISE 


0,54 


DISCUSSION 




MEDIATOR 




KNOWLEDGE 




OPENNESS 




COOPERATION 




ENGAGE 




PREVIEWS 




FOCUSED 




CUES 




STRUCTURE 


0,44 


SEQUAL 




SACVTIVE 




SFOSTER 




FOSTER 




INDIVIDULIZE 




REACTION 




SDISCRETE 




RELATION 




AUTHENTIC 




MULTICONTEXT 




ACCAFFECT 




ACCPRIVAT 




TEACHERFOCUS 




DIFFICULTY 




PROBLEMPROC 




VARIABILITY 




NORMA^ALUE 




LEARNTECH 





0,80 

0,77 

0,68 

0,66 

0,57 

0,45 



0,43 



0,42 



0,41 



0,81 

0,79 

0,72 

0,63 



-0,44 
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0,59 
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0,64 
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0,81 




-0,37 


0,80 




0,44 




-0,41 



0,64 
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Results indicate that the second and the seventh factor represent unique problem 
solving variance, which is not already assessed in the teaching effectiveness dimensions. 
These two factors correspond to the dimensions "Constructivist Instruction" and "Real - 
World Embedding", which emerged in the first analysis. The dimension "PS Processes" 
breaks up and is partially represented in the factor called Structure/Clarity. 

So is there no relation between instructional effectiveness and a PS teaching approach? 
The answer is no. The following figure (Figure 2) shows that there are interesting 
relations, which are not depicted in correlations or factor analyses. 
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Figure 2: Constructivist Instruction and Effectiveness 



This scatter plot displays the dimension "Constructivist Instruction" plotted against the 
dimension Effectiveness, a global measure consisting of control, speed, rule clarity, 
organization and time-on-task. The empty field shows that there are no ineffective 
lessons in which constructivist instruction can be found. A reasonable level of effectivity 
may be regarded as a precondition for the reahzation of student discussion and 
cooperation in a constructivist learning environment. 
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Does Problem Solving Instruction directly improve learning in Mathematics? The 
Glerman longitudinal enhancement of the TIMS study enables us to look at both 
achievement and learning in Mathematics. 

Table 4: Correlations with Achievement, Learning and Interest Growth 





Achievement 


Interest Growth 




TIMSS Score 


Growth 7 to 8 


Mathematics 
as a subject 




(n=82 classrooms) 


(n=81 classrooms) 


(n=67 classrooms) 


PS Constructivist Instruction 
PS Problem Processing 
PS Real-World Embedding 


t 

f 
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f 
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0,27 ♦ 
) 
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Clarity of Rules 


0,20 


) 


) 


Organisation 


0,16 
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Control 


0,17 
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) 


Time-On-Task 


o;27 ♦ 


0,15 


j 


Speed 


0,33 ♦♦ 


0,23 ♦ 


j 


Teacher Focusedness 


0,16 




-0,22 



* p < .05, ** p < .01; coefficients below + / - 0.15 are omitted ( , ) 



A look at table 4 shows that PS Instruction assessed by our three dimensions neither 
related to the achievement level at grade 8 (the class average of the TIMSS score) nor 
does it influence the learning form grade 7 to grade 8. In comparison the teacher 
effectiveness dimensions prove as being effective in fostering learning. On the other 
hand there may be an indirect influence of problem solving. The dimension 
Constructivist Instruction is positively correlated to the development of the students 
interest in Mathematics as a subject. 

In what way are the facets of problem solving instruction related to classroom discourse 
and structural characteristics of instruction? The TIMSS-Video Study provides us with a 
lot of information on low-inference characteristics of instruction, classroom interaction 
and classroom discourse. Significant relations between the PS dimensions and such 
characteristics are displayed in the following transparency. 

Constructivist Instruction is positively related to sharing, group seatwork and tasks that 
involve inventive thinking during seatwork phases. Constructivist instruction goes along 
with more and longer utterances by the students and with more utterances in which a 
student or the teacher evaluates a preceding utterance. It is negatively related to 
chalkboard use and to "non-cooperative" individual seatwork. 
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Lessons high in Problem Processing show more interactions in the form of "one task / one 
situation" and more teacher utterances characterized as "information". In such lessons 
fewer interactions in the form of "one task / multiple situations" and fewer discipline- 
related teacher utterances are found. 

Real-World Embedding is positively related to the use of manipulatives or models. 
Lessons high on this dimension show more student generated solutions, more alternative 
solutions and a higher amount of teacher talk with a demonstration purpose. On the 
other hand these lessons are negatively related to the complexity of topic taught. 

Do the external video ratings based on a small video sample correspond to the students’ 
general view of instruction? To answer this question we looked at the correlations of the 
PS dimensions with a set of items from the TIMSS student background questionnaire, in 
which the students were asked "How often do the following things happen in your 
Mathematics classroom?". 



Table 5: Problem Solving and Student Perceptions 

How often do the following things happen in your mathematics classroom? 

TIMSS Background Questionnaire (response format: always often - pretty often - once in a while - never) 



N=82 


PS Constructivist 


PS Problem Processing 


PS Real-World 




Instruction 




Embedding 


teacher shows us, how to solve 
mathemat. problems 


0,29** 


0,24* 


0^3* 


copy notes from the board 

have a quiz or test 

work from worksheets on our own 


y 

-0,17 

y 


) 

) 

) 


-0,17 

) 


work on mathematical projects 


0,21 


) 


0,17 


work in pairs or in small groups 


0,33** 


0,19 


0,30** 


solve problems with everyday life 
things 


y 


0,28* 


0,26* 


discuss homework 


0,20 


-0,17 


) 



p < .05, ** p < .01; coefficients below + / - 0.15 are omitted ( , ) 



Results in table 5 show that the item "The teacher shows us how to solve Mathematical 
problems." has significant positive correlations with all three PS Dimensions (and all 
nine PS facets). Furthermore the first and third dimensions, Constructivist Instruction 
and PS Processing, are positively related to the Items "work in pairs or small groups". 
PS Processing and Real World Embedding are positively related to the "solve problems 
with daily life things". The pattern of correlations shows that the video ratings are 
related to the students general perception of their Mathematics instruction. 
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Summary 

To assess problem solving in Mathematics instruction nine high-inference observation 
items were derived from hteratvure. These items were used in combination with a 
classical observation system of teacher effectiveness to rate the 8th grade Mathematics 
lessons from TIMSS. Some of the PS facets can be regarded as rare events. The results 
from factor analyses show that the facets have a 3-dimensional structure. The problem 
solving variance in the instructional behavior is specific and is not already represented 
in the classical features of instruction. A reasonable level of teaching effectiveness can be 
regarded as a precondition for a constructivist PS approach. The PS dimensions are not 
directly related to achievement and learning. A constructivist PS approach is related to 
interest growth in Mathematics. The PS facets are related to lesson characteristics and 
aspects of classroom discourse. Correlations with the student ratings of frequencies of 
various instructional activities also support the vahdity of the PS facets. 
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